Thermodynamics of used punched tape: A weak 
and a strong equivalence principle 
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Abstract — -We study the repeated use of a monotonic recording 
medium — such as punched tape or photographic plate — where marks can 
be added at any time but never erased. (For practical purposes, also 
the electromagnetic "ether" falls into this class.) Our emphasis is on 
the case where the successive users act independently and selfishly, but 
not maliciously; typically, the "first user" would be a blind natural pro- 
cess tending to degrade the recording medium, and the "second user" a 
human trying to make the most of whatever capacity is left. 

To what extent is a length of used tape "equivalent" — for informa- 
tion transmission purposes — to a shorter length of virgin tape? Can we 
characterize a piece of used tape by an appropriate "effective length" 
and forget all other details? We identify two equivalence principles. The 
weak principle is exact, but only holds for a sequence of infinitesimal us- 
age increments. The strong principle holds for any amount of incremental 
usage, but is only approximate; nonetheless, it is quite accurate even in 
the worst case and is virtually exact over most of the range — becoming 
exact in the limit of heavily used tape. 

The fact that strong equivalence does not hold exactly, but then it 
does almost exactly, comes as a bit of a surprise. 

Keywords — Thermodynamics of write-once media, Equivalence prin- 
ciples for storage capacity of noisy medium 

Bob — a poor computer science student — has found, rum- 
maging through Alice's dump, a large amount of used 
punched tape "in good conditions" . He doesn't care for 
the data that is already on the tape: he would like to reuse 
■ the tape for storing his own data. He wants to be able 
, to use a standard tape read/punch unit, which can sense 
holes in the tape and punch new ones but not remove holes 
that are already there. Since holes already made cannot 
be undone, the storage density Bob can expect to achieve 
is less than with virgin tape, and will depend on the actual 
conditions of the tape. 

To what extent is a length of used tape "equivalent" — for 
information transmission purposes — to a shorter length of 
virgin tape? Are there any qualitative differences between 
tapes that have been used to different degrees, or can one 
characterize a piece of used tape simply by its "effective 
length" and forget all other details? 

The theme we develop is complementary to that of Rivest 
and Shamir [7] (also cf. [HI). They stress the information- 
engineering aspects of reusing a tape generated by a co- 
operative partner in a pre-planned context. On the other 
hand, we are interested in a situation where the other party, 
while presumed non-malicious, volunteers no cooperation 
and pursues independent goals (if any goals can be made 
out); what we typically have in mind for "the other party" 
is natural processes. 1 

Tommaso Toffoli (tt9bu.edu), ECE Department, Boston Univer- 
sity, 8 Saint Mary's St., Boston, MA 02215. 

x As humans become more proficient at exploiting physical mech- 
anisms on a finer and finer scale for computational purposes, com- 
putation will look more and more like an attempt to encroach on 
a turf already jammed near capacity by heavy "native" traffic — the 
near-equilibrium bustle of microscopic matter (cf. Dyson|I]). The 
present study is part of a wider program aimed at exploring this kind 



The cumulative channel capacity of randomly-punched 
used tape was first investigated in |S] (also see references 
therein), some of whose results we simplify and extend. 
References and [H] discuss coding algorithms that dy- 
namically adjust to "stuck-at-0" faults on the tape (cells 
that will not punch) sensed during punching, and "stuck- 
at-1" faults sensed during or before punching. A paper re- 
lated to the present one in spirit if not in detailed substance 
is "Writing on dirty paper" by Costa[2] , whose moral ( "Do 
the best with what you have") we make our own. 

If you have no time at all, read just f|V] — a self-contained, 
intuitive debriefing. 

I. Orientation 

Each position on the tape where a hole may appear is 
called a cell; the two possible cell states are hole and blank. 
The instructions to the punch unit are punch and spare, 
with the following results on the tape 
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A hole (or punch) distribution that factors into identi- 
cal independent distributions for the individual cells — and 
is thus characterized by a single number, namely, the hole 
(or punch) density — will be called canonical. We shall as- 
sume that on each round of usage or stage the tape starts 
with a canonical hole distribution of density p and comes 
out with a uniform hole density p'\ furthermore, we as- 
sume that the intervening punching process packs on the 
tape the maximum amount of new information compatible 
with startng density p and target density p'. According to 
Shannon's theorem, such maximum efficiency can asymp- 
totically be achieved by means of sufficiently long block 
codes. From the above assumptions one can prove that 
both the punch distribution q yielded by an optimal code 
and the resulting hole distribution p' must be canonical 
as well. Thus, our usage assumptions imply that, starting 
from virgin tape — whose distribution is, of course, canon- 
ical with p = — input, punch, and output distributions 
will be canonical at every successive stage. For this reason, 
in what follows all distributions will be tacitly understood 
to be canonical. 

The result of applying a punch density q to a hole density 
p is a new hole density 



p' = l-(l-p)(l-«). 

of computational regime. 



(1) 
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A canonical punch distribution entails that, once the input 
hole density p is known, there is no further advantage in 
knowing the position of the individual holes; in other words, 
overpunching can be carried out in a data-blind fashion. 

Let's examine a few distinguished cases. 

• If p = the tape is blank — Bob can resell it as virgin tape. 

• If p = 1 — p = 1/2, the tape has already been utilized 
by Alice at its maximum information capacity of one bit 
per cell. That would seem to leave Bob with no room for 
further information storage. But remember that he doesn't 
care about the old information: punching new holes will 
destroy some of it but will encode some of his own! In fact 
(see 91111 below^l. with a punch density q = 3/5, Bob can 
record on the tape as much as about .322 bits per cell. 

• If p = 1, the tape carries no information for Alice — just 
as in the case p — 0. However, now there is no way Bob 
can put any information on it. Alice wantonly spoiled the 
tape. 

II. Notation 

If p is a probability, it will be convenient to write p for 
1 - p. Thus, in 0J, 



P 



l — pq=pq, or q=p'/p- 



We shall use natural logarithms throughout. It will be 
convenient to write tnx for — lnx. The self-information 
function, defined as 

y = xm~x, 

will play an important role in the equivalence principles 
discussed here (see 9VIII) . The binary entropy function, 

defined by 

H(jp) = pm~p + pm~p, 

is the average of the self-information function over the bi- 
nary distribution {p, p}. 

Both self-information and binary entropy, as defined 
here, measure information in natural units or nats. Con- 
version of information quantities to binary units or bits is 
achieved by explicitly factoring out the constant 



bit = In 2 



.693: 



thus, for example, the entropy of four equally probable mes- 
sages is In 4 = 2 In 2 = 2 bit. 

If X and Y are random variables, P(x) will denote the 
probability that X = x, and P(x.y) the probability that 
X = x and Y — y. The mutual information between X 
and Y is defined as 



{X;y} = ^P(^)In^ 



P{x)P{y) 
■y) ' 



For more background on information theory, see the ex- 
cellent introduction by Abramson 1 . 

III. Used tape as a monotonic binary channel 

Under the above assumptions (jflj, used punched tape 
may be viewed as a communication channel affected by 



monotonic noise. In the channel diagram of Fig. ^ the 
input variable X represents the instruction given to the 
punch unit while scanning a cell, and the output variable 
Y represents the resulting cell state. An "error" occurs 
when a cell spared by the punch unit turns out already to 
contain a hole. The conditional probability P(hole|spare) 
associated with this transition equals the current hole den- 
sity p. 
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Fig. 1 

Channel diagram of used punched tape viewed as a 
monotonic- error binary channel. 

From the joint and marginal distributions of X and Y, 
namely, 
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one obtains, for this channel operated at a punch density 
q, a mutual information 

AI = H(pq)-qH(p) = H(p')-^H(p). (3) 

The quantity AI is the amount of new information that can 
be encoded on a tape having a hole density p by punching it 
with a density q, resulting in a new hole density p'(p, q) = 
1 - pq. 

The relation expressed by equation |2J) — plotted in 
Fig. — completely characterizes the bulk properties of 
punched tape as a communication channel. The rest of this 
paper is devoted to extracting some of its implications. 

The capacity C of the channel is the maximum of AI 
over all possible values of q (or, equivalently, of p'). By 
equating to zero the derivative of AI with respect to p' , 



dAI _ p' 
= m = 



m =0i 
p 



dp' p 

one finds that this maximum occurs at 
1 



1 



- , or p' 



1 



e -H(p)/p + i' 



p(e H (p)/p + 1)' 
where AI attains the value 

C = \n(e- H(p)/ P + 1) = hip', 
as plotted in Fig. 03 In particular, for p = 1/2, 
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and C = In • 



.322 bit. 
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Mutual information AI of the "used tape" channel, plotted 

AS A FUNCTION OF THE PUNCH DENSITY q FOR VARIOUS VALUES OF 
THE CURRENT HOLE DENSITY p TREATED AS A PARAMETER. THE 
MAXIMUM OF EACH CURVE IS MARKED. 




Fig. 3 

Channel capacity of used tape, C, as a function of the 

CURRENT HOLE DENSITY p; THE DOTS MATCH THOSE OF FlG.[3] 



IV. Cooperation and competition 

For sake of contrast with the current context of self- 
ish, independent utilization of the tape by each successive 
party, in the following two subsections we shall briefly dis- 
cuss the possibilities of cooperation and competition. 

A. You shall receive an hundredfold 

In two successive selfish transmission stages starting 
from virgin tape, Alice got 1 bit's worth of message out 
of each cell and Bob .322 bit, for a total of 1.322 bit. By 
collaborating, they could do much better[7], 0- In fact, if 
Alice and Bob worked in concert, with a very simple code 
they could each get two bits' worth of message out of ev- 
ery three cells, for a total of 4/3 rj 1.333 bit/cell; with 
long block codes, they could get up to about 1.55 bit/cell. 
The advantages of collaboration show up even better when 
one can plan ahead a long series of transmission stages 
with a long length of tape: in this situation, the cumula- 
tive amount of message worth one can get out of an n-cell 



q 



~ 1 1 

q ~ p{e H M/p + l) 






, — y. 







1/2 1 p — > 

Fig. 4 

Plot of the optimal punch density q vs current hole density 
p. The distinguished points are discussed in the text. That 
lying on THE q = p line is discussed in illV-BI 



length of tape grows as nlnn; therefore, the amount per 
unit length is unbounded! 

B. Tape wars averted 

We have seen that, if the original tape was punched at 
a density p by Alice, with an attendant rate H(p) for her 
message, then Bob can achieve his channel capacity C(p) 
as in (JSJ by punching at density q(p) as in l|4"|. In the 
process, Alice's original message is, of course, degraded. In 
fact, if Alice tries to read back her message, she will find 
it contaminated by the same amount of one-way noise as if 
it had gone through the channel described by exchanging 
q and p in Fig. and table J3J. 

Suppose now that Alice, realizing that her tapes are go- 
ing to be reused (or concurrently used — since, as we have 
seen, the two punching operations commute) by Bob, de- 
cides to encode her next batch of tapes so as to make her 
messages readable even after an anticipated punching by 
Bob at density q. According to 10} and Fig. as a pre- 
ventive measure she will have to shift her punch density p 
to a higher value than 1/2, thus achieving a lower rate but 
greater resistance to Bob's tampering. When Bob realizes 
that, he will be forced to shift his punch density q to a 
higher value — and so forth. 

This is not a zero-sum game: as the arms race unfolds, 
each party will end up storing progressively less informa- 
tion on the tape. Will the race lead to the mutual de- 
struction of information capacity? Fortunately, the curve 
of Fig. intersects the line q = p and there has a slope less 
than 1. Thus, the race converges to a stable point (with 
q = p w 0.609), where each party achieves an effective 
storage capacity of « .240 bit/cell. 

The sum of the two capacities — and these are coexist- 
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ing capacities, with both messages readable at the same 
time! — is about 0.48 bit/cell, to be compared with the 
1 bit/cell Bob and Alice could have achieved by "space- 
sharing" the tape (e.g., one cell for Alice, one for Bob, 
and so forth). Thus, the attempt by the two parties to 
concurrently use the monotonic-write tape, performed in a 
selfish but rational way, results in an overall loss of storage 
capacity that is substantial but not crippling. 

It must be noted that, even though at equilibrium they 
are in a symmetric situation, Alice and Bob cannot use 
the very same block code to encode their messages on the 
tape. To avoid interference the two codes must be practi- 
cally uncorrelated or "mutually orthogonal" ; this is always 
possible with long enough block codes. 

V. Intercom dialogues 

We introduce the issue of tape equivalence by means of 
two dialogues. The length £ of a piece of tape is the number 
of cells it contains. Because of the canonical distribution 
of both holes and punches (J|J, the overall capacity of a 
tape of length £ is £ times the capacity of a single cell, and 
similarly for the mutual information. 

Dialogue 1 

Bob is now an old and stingy facilities officer at Caltech. 
He can no longer see the individual holes on the tape — 
his vision is blurred — and he wouldn't any longer know 
how to start writing a block code. All he cares about is 
tape as a bulk commodity, and getting the most out of 
it. He is assisted by Sue, who physically handles the tape 
and knows how to devise appropriate block codes. Sue has 
standing instructions to recycle paper tape to the best of 
her capabilities and not to bother Bob with details. 

Bob, on the intercom: Sue, we have to send a million-bit 
message to MIT. Get a piece of tape. 

Sue, from the mail room: I've got here a reel of tape with 
an overall capacity of one million bits. [She doesn't tell 
Bob whether that's a thousand feet of virgin tape, or 
perhaps ten thousand feet of heavily used tape.] 

Bob: Good! Here is the message. Don't waste any capac- 
ity, and make sure you get the tape back from MIT so 
we can reuse it! By the way, what will be the capacity 
left on the tape after this message? I want to enter it as 
an asset in my inventory sheet. 

Sue: That will be 333,000 bits. 

Bob: So, using this tape at capacity will leave it "shrunk" 
to .333 of its previous capacity. Well, one third left is 
better than nothing! 

Bob, a week later: Sue, here is another message for MIT. 
Since it happens to be 333,000 bits long, let's use the tape 
you got back from them. [We assume that, after decod- 
ing a message, MIT does not keep a record of the detailed 
hole pattern received. That might be used for improving 
transmission efficiency, but at substantial storage cost.] 
What will be the capacity left on the tape after this mes- 
sage? 



Sue: That will be about 119,000 bits. 

Bob, punching keys on a calculator: Hey, this time, 
it will only shrink to 119,000/333,333=.357 of its pre- 
transmission capacity [with an accusing tone] Are you 
sure you made the best use of my tape last time? 

Sue, chuckling: Cool off, Bob! Every housewife knows 
that used tape shrinks less! In fact, really ripe tape only 
shrinks to 1/e « .368 of its previous capacity upon each 
usage. 

Bob: And brand new tape? 

Sue: New tape is the worst! It will shrink to In 5/ In 2 — 2 w 
.322 of its previous capacity. Here's the whole picture! 
(Fig. 01 
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"Shrinkage coefficient" s = C(p')/C(p) for successive 

FULL-CAPACITY USAGES (LABELED 1,2, .... oo) OF AN INITIALLY 
VIRGIN TAPE. AS THE TAPE GETS MORE THOROUGHLY USED, THE 
SHRINKAGE COEFFICIENT RAPIDLY CONVERGES TO 1/e. 



The two physical parameters of a piece of tape, namely, 
its length £ (in cells) and its current hole density p, com- 
pletely characterize its "response" — in terms of amount of 
information transmitted and capacity left — upon each suc- 
cessive usage, including usages with a punch density q < q 
(where some of the capacity is saved for later) or q > q 
(where some capacity is wasted), according to equations 
JTJ, ®, and 0. 

In particular, the capacity of a piece of tape of length £ is 
£C(p) (cf. J5J). This can be thought of — if we measure ca- 
pacity in bits (see JIIJ — as the reduced length of the tape — 
i.e., the number of cells of virgin tape having the same over- 
all capacity. Bob would have been delighted to find that 
two pieces of tape having the same reduced length are com- 
pletely equivalent for information-transmission purposes. 
Such an equivalence principle would allow him to charac- 
terize a piece of tape by means of a single information- 
theoretical parameter — the reduced length — rather than 
the two physical parameters £ and p, and greatly simplify 
his inventory bookkeeping. 

If such an equivalence held, then, as a specific conse- 
quence, the shrinkage coefficient of Dialogue 1, defined as 



s(p) 



C(P)' 



would be independent of p. Unfortunately, as we have seen 
in the dialogue, this is only approximately true (Fig. |5j). 
We'll return to this problem, with better tools, in Will 



Dialogue 2 

Sue is on vacation. Her temporary replacement, Willie, 
is being indoctrinated by Bob about the need to conserve 
tape. To test his coding capabilities, Bob chooses a spool 
of tape just like the one he gave Sue the first time. 

Bob: Here is a length of used tape, Willie, and a million-bit 
message to be sent to MIT. Please transmit the message 
as efficiently as you can. 

Willie: Is it urgent? 

Bob: Not, really. Take your time, but do a good job! 

Bob, a month later: Well, did you get the tape back from 
MIT? 

Willie: Here it is! 

Bob: What's its capacity now? 

Willie: 580,000 bits, more or less. 

Bob: What? It only shrank to .580 of its original length? 
How did you manage that? 

Willie: You know, haste makes waste. So I first encoded 
only a small fraction of the message on the tape, using 
a very low punch density. MIT decoded that, wrote it 
down, and sent back the tape. Then I encoded on the 
same tape another increment of the message, sent it to 
MIT, and so on. The tape must have gone back and 
forth twenty times! 

Bob: In the limit of an infinite number of infinitesimal 
increments, how much information could you transmit 
in this way? 

Willie: Starting from virgin tape, about 2.37 bits/cell 

(precisely, 
Bob: That's amazing! 

Willie: And, of course, at any intermediate moment the 
transmission "mileage" already used plus that which is 
still left on the tape equals a constant — provided you 
always travel very slowly. 

Bob: I got it! Your "mileage left" is the effective length I 
was looking for. No matter how different they look phys- 
ically, two pieces of tape (say, one short and fresh and 
the other long and stale) having the same effective length 
are equivalent for information transmission purposes. 

Willie: Slow down, Bob! That is true only as long as you 
use them up slowly. By comparing Sue's performance 
with mine, you realize that, when one tries to cram onto 
a tape a substantial fraction of its channel capacity at 
once, there are losses by "friction", as it were. 2 Well, one 

2 This behavior is qualitatively similar to that of mechanical sys- 
tems. Consider a battery of internal resistance R connected to a load 
of impedance r. It will be convenient to use the normalized variable 
p = 1/(1 + r/R), which goes from to 1 as r/R goes from oo to 0. 
The maximum power transfer occurs when p = 1/2 (i.e., r = R); in 
this case, half of the energy is dissipated by friction in R. As p —* 0, 
energy is transfered to the load more slowly but less of it is wasted 
by friction. (As p — > 1, one gets less power out and wastes a greater 
fraction of the energy.) Indeed, to an untrained eye the power trans- 
fer curve 2p(l — p) — an inverted parabola — is hard to tell apart from 
the binary entropy curve H(p). 



can tell the difference between fresh tape and well-worn 
tape by the fact that the former exhibits just a little 
more friction than the latter. 

VI. Weak equivalence 

Let us explore in more detail what Bob discovered with 
Willie's help. 

Suppose that we start with virgin tape and record on it 
a small amount dl of information by punching it at a very 
low density. We ship the tape but ask the recipient to send 
it back to us after reading the message. We then record 
on this "slightly used" tape an additional small amount 
of information, 3 further increasing the hole density. We 
continue in this way, sending one after the other a large 
number of messages each having a small information con- 
tents, until the tape is completely filled with holes. If at 
each stage the encoding is done optimally, what is the cu- 
mulative information J dl of the messages we sent? 

Assume that at a generic stage of this process we start 
with a hole density p and increase it to p' = p + dp by 
issuing punch commands with a probability dq per cell. 
The channel diagram is the same as Fig. Q but with input 
and output probabilities as in Fig. 
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Fig. 6 

Channel diagram for used tape, with input and output 
probabilities corresponding to incremental use of the 
channel capacity. 

The hole density increment is dp = pdq, as the blanks, 
which appear with density p, are turned into holes with 
probability dq, while the holes, with density p, remain unaf- 
fected. The mutual information of this infinitesimal punch- 
ing operation, calculated from J2J using dq in place of q, 
is 



dl = H{p + dp) - E±j!P.H(p) 



In — H — — | dp — -^^dp. 



P 



P 



P 



(6) 
(7) 



The indefinite integral of the integrand in the last expres- 
sion is 

hT-P 

dp = -Li 2 (l - p), 

1 -p 

where Li2 is the dilogarithm function. 4 Thus, the effective 
capacity of a tape of hole density p, i.e., the total amount 

s For this, we need, of course, a very long block code. 

4 This is one of the polylogarithm functions, defined by Li n (z) = 

Eoo z k 
k=l k n ' 
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of information that can be transmitted via it in successive 
small increments until all holes have been punched up, is 



Q(p) = 



i i 

dI=-U 2 (l-x) =Li 2 (p), (8) 



V 



as plotted in Fig. \7\ (compare with the qualitatively similar 
behavior of C, in Fig. 0); for virgin tape (p = 0), the 
effective capacity is Li2(l) = 7r 2 /6 (cf. 0). Note that, by 



dl 



-dQ. 



Since Q is a function of state of the tape (i.e., it depends 
only on its state and not on the specific sequence of oper- 
ation that led to that state), dl is an exact differential. 




Fig. 7 

Effective capacity Q as a function of the current hole 

DENSITY p. 

The above quantities are on a per-cell basis. Let us de- 
fine the effective length (cf. Dialog 2) of a piece of tape of 
length £ and hole density p as A = £Q(p). If by a sequence 
of small incremental messages we transmit an amount of 
information / per cell, and thus a total amount S = It 
for the entire piece of tape, the new effective length will 
be A' = £(Q — I). The corresponding shrinkage coefficient 5 
will be 

^ = 1-1 = 1-* 
A Q X 

which is independent of the physical parameters £ and p 
and depends only on the ratio between two information- 
theoretical quantities, i.e., the total amount S of informa- 
tion transmitted and the effective length A of the tape. We 
shall call this the weak equivalence principle for monotonic- 
write media. 

VII. Strong equivalence 

Let us now explore in more detail what Bob discovered 
with Sue's help. 

Whether we intend to utilize a piece of tape incremen- 
tally, as in Dialogue 2, or in discrete installments, as in 
Dialogue 1, the effective length A defined above provides 

5 This quantity is analogous to but distinct from the shrinkage co- 
efficient of Dialogue 1, which is a ratio of channel capacities. 



a more natural measure of a tape's information capacity 
than the reduced length introduced in Dialogue 1. 

Armed with this measure, let us now turn our attention 
from the special case of the limit of an infinite sequence 
of infinitesimal messages to the general case of finite-size 
messages, where the weak principle is not applicable. 

Our goal is to eliminate the physical parameters p and 
q between equations Q and (jSJ, and thus write a relation 
directly between (a) the effective length A of a piece of 
tape before the transmission of a message, (b) the effective 
length A' after the transmission, and (c) the amount S of 
information conveyed by the message. If such a relation 
exists, it may be assumed to be of the form 

f(X,X',S) = 

and, since we are assuming a canonical hole distribution 
before and after punching, it must satisfy the scaling prop- 
erty 

f(aX, aX 1 , aS) = for any a. 

Setting, as a special case, a = 1/A', we obtain a relation 
between two variables 



where 



_ A' _ Q(p') 
° A Q{p) 



A S I(p,q) 



The variable p — which is the mutual information for a 
given stage of utilization of the tape — can be thought of 
as the information rate per unit of effective length of the 
tape, and a as the shrinkage coefficient attendant to that 
stage. 

Since the variables a and [i depend on two parameters, p 
and q, we cannot a priori expect to eliminate both param- 
eters when solving for [i with respect to a. However, for 
a given initial hole density p treated as a fixed parameter, 
we can eliminate just q and write 

H = n P (a). 

The result of this elimination, performed numerically for 
different values of p, are shown in Fig. [HI which also shows 
the values of the eliminated parameter q on the ^(c) curves. 

Paralleling the weak equivalence principle of the previous 
section — which states that tapes having the same effective 
capacity are indistinguishable at slow utilization rates — a 
strong equivalence principle would be one that is valid for 
any rate of utilization of the tape at any transmission stage, 
from an infinitesimal hole-density increment (q close to 0) 
to gross overpunching (q close to 1). I don't know whether 
it is more surprising that, strictly speaking, punched tape 
does not obey a strong equivalence principle, or that, after 
all, it turns out to do so to a very good approximation. In 
fact, as is clear from Fig. |H1 after eliminating q between /i 
and <r some dependence on p remains, but this dependence 
is slight in any case and rapidly vanishes as p approaches 
1. Intuitively, the one-parameter family of curves of Fig. [21 
nearly collapses — when expressed in terms of a more nat- 
ural set of variables — onto a single curve (Fig. |HJ. 
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Fig. 8 

Mutual information per unit of effective length, u., as a function of the shrinkage coefficient a, for. different values of 

THE INITIAL HOLE DENSITY p. THE DOTTED LINES REPRESENT LOCI OF EQUAL VALUES FOR THE RUNNING PARAMETER q. FOR ANY VALUE OF p, 
THE MAXIMUM OF fjt REPRESENTS THE CORRESPONDING CHANNEL CAPACITY GIVEN IN UNITS OF EFFECTIVE LENGTH. THE MAXIMUM OF flo 
OCCURS AT K = |(1 — In 2 2). FOR UTILIZATION RATE BELOW CAPACITY, THE SAME MUTUAL INFORMATION CAN BE OBTAINED WITH TWO 

DIFFERENT VALUES OF SHRINKAGE, ONE CORRESPONDING TO RATIONAL USAGE OF THE TAPE AND THE OTHER TO NEEDLESSLY WASTEFUL USAGE. 



The curves h p ((j) all have slope —1 at a = 1; this is an 
expression of the weak equivalence principle (i.e., for small 
q, the effective length decreases by an amount equal to the 
amount of information transmitted). They all have slope 
oo at er = 0, signifying that the waste of effective capac- 
ity increases precipitously when one punches at a density 
much greater than that needed for transmitting at channel 
capacity. 

The worst-case departure of the [i v curves from the lim- 
iting curve /ii = linip^i fi p occurs near the maximum bulge 
of the curves, and is substantially the same as the depar- 
ture of s from its 1/e limit as plotted in Fig. EI The curves 
Hp are not likely to be expressible in closed form; however, 
as is easy to prove, the limiting curve /ii is nothing but the 
familiar self-information function fi = a In a. To the same 
approximation as the strong equivalence principle holds, 
this function gives the information-transfer characteristics 
of punched tape (i.e., for any message, the capacity used 
by it, that wasted, and that left after the message) over the 
tape's entire utilization range. 

Let us remark that the self-information function appears 
in the limit also in Fig. [5] In fact, one can show that 



lim 

p— l 



C(p) 



eqtaq. 



VIII. Conclusions 

A piece of randomly punched tape is described by two 
physical parameters — its length I and its hole density p. 
We have raised the question of whether the tape's be- 
havior as an information transmission commodity can be 
usefully characterized by a single information-theoretical 
parameter — its effective length A. We have concluded that 
this is the case 



• in the "quasi-static" limit of slow utilization rate (weak 
equivalence principle) ; 

• for any utilization rate (strong equivalence principle) 

— exactly, but only in the limit of already heavily used 
tape, and 

— approximately — but with good accuracy even in the 
worst case — over the whole range of previous and future 
uses of the tape. 
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